On Fri, Oct 3, 2025, at 19:34, Julia Evans via GitGitGadget wrote:
> From: Julia Evans <julia@jvns.ca>
>
> Git very often uses the terms "object", "reference", or "index" in its
> documentation.
>
> However, it's hard to find a clear explanation of these terms and how
> they relate to each other in the documentation. The closest candidates
> currently are:
>
> 1. `gitglossary`. This makes a good effort, but it's an alphabetically
>     ordered dictionary and a dictionary is not a good way to learn
>     concepts. You have to jump around too much and it's not possible to
>     present the concepts in the order that they should be explained.
> 2. `gitcore-tutorial`. This explains how to use the "core" Git commands.
>    This is a nice document to have, but it's not necessary to learn how
>    `update-index` works to understand Git's data model, and we should
>    not be requiring users to learn how to use the "plumbing" commands
>    if they want to learn what the term "index" or "object" means.
> 3. `gitrepository-layout`. This is a great resource, but it includes a
>    lot of information about configuration and internal implementation
>    details which are not related to the data model. It also does
>    not explain how commits work.
>
> The result of this is that Git users (even users who have been using
> Git for 15+ years) struggle to read the documentation because they don't
> know what the core terms mean, and it's not possible to add links
> to help them learn more.
>
> Add an explanation of Git's data model. Some choices I've made in
> deciding what "core data model" means:
>
> 1. Omit pseudorefs like `FETCH_HEAD`, because it's not clear to me
>    if those are intended to be user facing or if they're more like
>    internal implementation details.
> 2. Don't talk about submodules other than by mentioning how they
>    relate to trees. This is because Git has a lot of special features,
>    and explaining how they all work exhaustively could quickly go
>    down a rabbit hole which would make this document less useful for
>    understanding Git's core behaviour.
> 3. Don't discuss the structure of a commit message
>    (first line, trailers, GPG signatures, etc).
>    Perhaps this should change.
>
> Some other choices I've made:
>
> 1. Mention packed refs only in a note.

I don’t think it’s worth mentioning this at all.  More on that later.

> 2. Don't mention that the full name of the branch `main` is
>    technically `refs/heads/main`. This should likely change but I
>    haven't worked out how to do it in a clear way yet.

I think this is worth getting into.  This is a pretty
user-facing concept.

> 3. Mostly avoid referring to the `.git` directory, because the exact
>    details of how things are stored change over time.
>    This should perhaps change from "mostly" to "entirely"
>    but I haven't worked out how to do that in a clear way yet.

I think that’s good.  I mean, I think us users don’t need that level of
detail and shouldn’t be “inspired” to muck with the internals.  If that
makes sense.  (See later)

>
> Signed-off-by: Julia Evans <julia@jvns.ca>
> ---
>     doc: Add a explanation of Git's data model
>[snip]
> diff --git a/Documentation/Makefile b/Documentation/Makefile
>[snip]
> diff --git a/Documentation/gitdatamodel.adoc
> b/Documentation/gitdatamodel.adoc
> new file mode 100644
> index 0000000000..4b2cb167dc
> --- /dev/null
> +++ b/Documentation/gitdatamodel.adoc
> @@ -0,0 +1,226 @@
> +gitdatamodel(7)
> +===============
> +
> +NAME
> +----
> +gitdatamodel - Git's core data model
> +
> +DESCRIPTION
> +-----------
> +
> +It's not necessary to understand Git's data model to use Git, but it's
> +very helpful when reading Git's documentation so that you know what it
> +means when the documentation says "object" "reference" or "index".

I haven’t gone hunting through the docs to see if this is covered
elsewhere.  But the thrust of all the things here definitely feel to me
like something that should be presented and documented in such a way.

> +
> +Git's core operations use 4 kinds of data:

Maybe small numerals should be spelled as words in running text?

> +
> +1. <<objects,Objects>>: commits, trees, blobs, and tag objects
> +2. <<references,References>>: branches, tags,
> +   remote-tracking branches, etc
> +3. <<index,The index>>, also known as the staging area
> +4. <<reflogs,Reflogs>>

Reflogs is certainly auxiliary ref data. What makes it qualify as
one-of-the-four?  I am open to it being both, to be clear.

> +
> +[[objects]]
> +OBJECTS
> +-------
> +
> +Commits, trees, blobs, and tag objects are all stored in Git's object
> database.
> +Every object has:
> +
> +1. an *ID*, which is the SHA-1 hash of its contents.
> +  It's fast to look up a Git object using its ID.
> +  The ID is usually represented in hexadecimal, like
> +  `1b61de420a21a2f1aaef93e38ecd0e45e8bc9f0a`.
> +2. a *type*. There are 4 types of objects:
> +   <<commit,commits>>, <<tree,trees>>, <<blob,blobs>>,
> +   and <<tag-object,tag objects>>.
> +3. *contents*. The structure of the contents depends on the type.
> +
> +Once an object is created, it can never be changed.
> +Here are the 4 types of objects:

As a curious Git user this seems correct.

> +
> +[[commit]]
> +commits::
> +    A commit contains:
> ++
> +1. Its *parent commit ID(s)*. The first commit in a repository has 0
> parents,

Maybe this is a subjective style thing but is it necessary to use “(s)”
when the context makes clear that it could be zero to many?

    Its *parent commit IDs. ...

> +  regular commits have 1 parent, merge commits have 2+ parents

s/2+/two or more/ ?

Same point as the “numeral” one above.

> +2. A *commit message*
> +3. All the *files* in the commit, stored as a *<<tree,tree>>*
> +4. An *author* and the time the commit was authored
> +5. A *committer* and the time the commit was committed
> ++
> +Here's how an example commit is stored:
> ++
> +----
> +tree 1b61de420a21a2f1aaef93e38ecd0e45e8bc9f0a
> +parent 4ccb6d7b8869a86aae2e84c56523f8705b50c647
> +author Maya <maya@example.com> 1759173425 -0400
> +committer Maya <maya@example.com> 1759173425 -0400
> +
> +Add README
> +----
> ++
> +Like all other objects, commits can never be changed after they're
> created.
> +For example, "amending" a commit with `git commit --amend` creates a
> new commit.

> +The old commit will eventually be deleted by `git gc`.

Maybe this could be moved to a part about what happens (eventually) to
unreachable objects?

Mentioning `git gc` and how things will get deleted raises
questions naturally. Like why would they be deleted? Okay
that’s clear: the previous commit will be replaced by the
amended one. Then when it is not reachable by anything
(even the reflog) it will get garbage collected.

It all follows. But is the reader necessarily mature enough
in their understanding to make the inference?

This is a long-winded way of saying: if you’re gonna discuss
`git gc` you might need to go into all of these concepts.

> +
> +[[tree]]
> +trees::
> +    A tree is how Git represents a directory. It lists, for each item
> in
> +    the tree:
> ++
> +1. The *permissions*, for example `100644`
> +2. The *type*: either <<blob,`blob`>> (a file), `tree` (a directory),
> +  or <<commit,`commit`>> (a Git submodule)
> +3. The *object ID*
> +4. The *filename*
> ++
> +For example, this is how a tree containing one directory (`src`) and
> one file
> +(`README.md`) is stored:
> ++
> +----
> +100644 blob 8728a858d9d21a8c78488c8b4e70e531b659141f README.md
> +040000 tree 89b1d2e0495f66d6929f4ff76ff1bb07fc41947d src
> +----
> ++
> +*NOTE:* The permissions are in the same format as UNIX permissions, but
> +the only allowed permissions for files (blobs) are 644 and 755.
> +

Makes sense.

> +[[blob]]
> +blobs::
> +    A blob is how Git represents a file. A blob object contains the
> +    file's contents.
> ++
> +Storing a new blob for every new version of a file can get big, so
> +`git gc` periodically compresses objects for efficiency in
> `.git/objects/pack`.

This gets into mentioning implementation files(?) like you mentioned in
the commit message.

1. That it’s a packfile and where it is might be too much detail for
   this doc
2. I vaguely recall documents discussing what happens to “storing every
   version” discussing deltas instead of packs? Again, I am not a Git
   developer though.

> +
> +[[tag-object]]
> +tag objects::
> +    Tag objects (also known as "annotated tags") contain:
> ++
> +1. The *tagger* and tag date
> +2. A *tag message*, similar to a commit message
> +3. The *ID* of the object (often a commit) that they reference

s/often/typically/ ?

I know it can get tedious to caveat the 99% cases with things that are
technically possible.  Maybe if it gets “bad enough” there could be a
part that explains/distinguishes the high-level/porcelain Git use and
what is technically possible: you make a `git tag -a`, which is on a
commit... except if you accidentally run it on top of an existing
tag. Then even the porcelain won’t protect you from making a 
tag-on-tag. (But it will issue a warning I guess.) Hmm. Now I don’t know.

> +
> +[[references]]
> +REFERENCES
> +----------
> +
> +References are a way to give a name to a commit.
> +It's easier to remember "the changes I'm working on are on the `turtle`
> +branch" than "the changes are in commit bb69721404348e".
> +Git often uses "ref" as shorthand for "reference".

Good.

> +
> +References that you create are stored in the `.git/refs` directory,
> +and Git has a few special internal references like `HEAD` that are
> stored
> +in the base `.git` directory.

Implementation file details.

You also mention `.git/refs/heads/<name>` below.  But refs aren’t stored
as files if you are using the *reftable* backend.  And that backend will
become the default for new repositories in Git 3.0, I think.

How does reftable work?  I don’t know.  But I don’t think we need to
know after reading this doc. :)

To be clear: how files are stored might not matter here.

> +
> +References can either be:
> +
> +1. References to an object ID, usually a <<commit,commit>> ID
> +2. References to another reference. This is called a "symbolic
> reference".

You seem to have used `**` when introducing terms:

    This is a *symbolic reference*

>[snip ref stuff]
> +
> +[[HEAD]]
> +HEAD: `.git/HEAD`::
> +    `HEAD` is where Git stores your current <<branch,branch>>.
> +    `HEAD` is normally a symbolic reference to your current branch, for
> +    example `ref: refs/heads/main` if your current branch is `main`.
> +    `HEAD` can also be a direct reference to a commit ID,
> +    that's called "detached HEAD state".
> +
> +[[remote-tracking-branch]]
> +remote tracking branches: `.git/refs/remotes/<remote>/<branch>`::
> +    A remote-tracking branch is a name for a commit ID.
> +    It's how Git stores the last-known state of a branch in a remote
> +    repository. `git fetch` updates remote-tracking branches. When
> +    `git status` says "you're up to date with origin/main", it's looking at
> +    this.

Looks good.

> +
> +[[other-refs]]
> +Other references::
> +    Git tools may create references in any subdirectory of `.git/refs`.
> +    For example, linkgit:git-stash[1], linkgit:git-bisect[1],
> +    and linkgit:git-notes[1] all create their own references
> +    in `.git/refs/stash`, `.git/refs/bisect`, etc.
> +    Third-party Git tools may also create their own references.
> ++
> +Git may also create references in the base `.git` directory
> +other than `HEAD`, like `ORIG_HEAD`.
> +

> +*NOTE:* As an optimization, references may be stored as packed
> +refs instead of in `.git/refs`. See linkgit:git-pack-refs[1].

I don’t know if this is relevant for both ref backends. And does it
matter?

> +
> +[[index]]
> +THE INDEX
> +---------
> +
> +The index, also known as the "staging area", contains the current
> staged
> +version of every file in your Git repository. When you commit, the
> files
> +in the index are used as the files in the next commit.
> +
> +Unlike a tree, the index is a flat list of files.
> +Each index entry has 4 fields:
> +
> +1. The *permissions*
> +2. The *<<blob,blob>> ID* of the file
> +3. The *filename*
> +4. The *number*. This is normally 0, but if there's a merge conflict
> +   there can be multiple versions (with numbers 0, 1, 2, ..)
> +   of the same filename in the index.
> +
> +It's extremely uncommon to look at the index directly: normally you'd
> +run `git status` to see a list of changes between the index and
> <<HEAD,HEAD>>.
> +But you can use `git ls-files --stage` to see the index.
> +Here's the output of `git ls-files --stage` in a repository with 2
> files:
> +
> +----
> +100644 8728a858d9d21a8c78488c8b4e70e531b659141f 0 README.md
> +100644 665c637a360874ce43bf74018768a96d2d4d219a 0 src/hello.py
> +----
> +
> +[[reflogs]]
> +REFLOGS
> +-------
> +
> +Git stores the history of branch, tag, and HEAD refs in a reflog
> +(you should read "reflog" as "ref log"). Not every ref is logged by

You’ve heard of the re-flog too?

> +default, but any ref can be logged.
> +
> +Each reflog entry has:
> +
> +1. *Before/after *commit IDs*
> +2. *User* who made the change, for example `Maya <maya@example.com>`
> +3. *Timestamp*
> +4. *Log message*, for example `pull: Fast-forward`
> +
> +Reflogs only log changes made in your local repository.
> +They are not shared with remotes.

Makes sense.

> +
> +GIT
> +---
> +Part of the linkgit:git[1] suite

I appreciate that this is the first version and you might have plans
after this one. But I wonder if this doc could use a fair number of
`gitlink` to branch out to all the other parts. Like git-reflog(1),
gitglossary(7).

Thanks for starting on a whole new doc. That must take quite
some effort.

"Julia Evans via GitGitGadget" <gitgitgadget@gmail.com> writes:

> +MAN7_TXT += gitdatamodel.adoc
>  MAN7_TXT += gitdiffcore.adoc
> ...
> +gitdatamodel(7)
> +===============
> +
> +NAME
> +----
> +gitdatamodel - Git's core data model
> +
> +DESCRIPTION
> +-----------

The above causes doc-lint to barf.

https://github.com/git/git/actions/runs/18265502271/job/51999236907#step:4:655

gitdatamodel.adoc:226: has no required 'SYNOPSIS' section!
    LINT MAN SEC giteveryday.adoc
make[1]: *** [Makefile:498: .build/lint-docs/man-section-order/gitdatamodel.ok] Error 1


You can check locally with "make check-docs" without waiting for my
integration cycle to push to GitHub CI.

Thanks.

> The above causes doc-lint to barf.
>
> https://github.com/git/git/actions/runs/18265502271/job/51999236907#step:4:655
>
> gitdatamodel.adoc:226: has no required 'SYNOPSIS' section!
>     LINT MAN SEC giteveryday.adoc
> make[1]: *** [Makefile:498: 
> .build/lint-docs/man-section-order/gitdatamodel.ok] Error 1
>
>
> You can check locally with "make check-docs" without waiting for my
> integration cycle to push to GitHub CI.


Thanks, will fix.

Thanks for the review!

>> 2. Don't mention that the full name of the branch `main` is
>>    technically `refs/heads/main`. This should likely change but I
>>    haven't worked out how to do it in a clear way yet.
>
> I think this is worth getting into.  This is a pretty
> user-facing concept.

I think I'll see if I can figure out a way to mention this and at the
same time remove most of the rest of the references to the `.git`
directory when explaining references (which you talked about
further down), including packed refs.

>> +
>> +1. <<objects,Objects>>: commits, trees, blobs, and tag objects
>> +2. <<references,References>>: branches, tags,
>> +   remote-tracking branches, etc
>> +3. <<index,The index>>, also known as the staging area
>> +4. <<reflogs,Reflogs>>
>
> Reflogs is certainly auxiliary ref data. What makes it qualify as
> one-of-the-four?  I am open to it being both, to be clear.

The reason I like to talk about reflogs is that it gives you a
way to "undo" Git operations that can be really useful. 
And any Git command that updates refs can updates that
ref's reflog.

Understanding how reflogs work helps to understand what the
limitations of using reflogs to undo mistakes is: for example
the index is not a ref, so you can't use the reflog to undo
changes to the index.

>> +2. A *commit message*
>> +3. All the *files* in the commit, stored as a *<<tree,tree>>*
>> +4. An *author* and the time the commit was authored
>> +5. A *committer* and the time the commit was committed
>> ++
>> +Here's how an example commit is stored:
>> ++
>> +----
>> +tree 1b61de420a21a2f1aaef93e38ecd0e45e8bc9f0a
>> +parent 4ccb6d7b8869a86aae2e84c56523f8705b50c647
>> +author Maya <maya@example.com> 1759173425 -0400
>> +committer Maya <maya@example.com> 1759173425 -0400
>> +
>> +Add README
>> +----
>> ++
>> +Like all other objects, commits can never be changed after they're
>> created.
>> +For example, "amending" a commit with `git commit --amend` creates a
>> new commit.
>
>> +The old commit will eventually be deleted by `git gc`.
>
> Maybe this could be moved to a part about what happens (eventually) to
> unreachable objects?
>
> Mentioning `git gc` and how things will get deleted raises
> questions naturally. Like why would they be deleted? Okay
> that’s clear: the previous commit will be replaced by the
> amended one. Then when it is not reachable by anything
> (even the reflog) it will get garbage collected.
>
> It all follows. But is the reader necessarily mature enough
> in their understanding to make the inference?
>
> This is a long-winded way of saying: if you’re gonna discuss
> `git gc` you might need to go into all of these concepts.

If folks here think this is a reasonable document to add to
Git I'll try get some beta readers to read this, see which parts
folks find confusing, and address those, keeping the `git gc`
stuff in mind.

Similarly for the style comments.

>> +blobs::
>> +    A blob is how Git represents a file. A blob object contains the
>> +    file's contents.
>> ++
>> +Storing a new blob for every new version of a file can get big, so
>> +`git gc` periodically compresses objects for efficiency in
>> `.git/objects/pack`.
>
> This gets into mentioning implementation files(?) like you mentioned in
> the commit message.

That's true! The reason I think this is important to mention is that I find
that people often "reject" information that they find implausible, even
if it comes from a credible source. ("that can't be true! I must be
not understanding correctly. Oh well, I'll just ignore that!")

I sometimes hear from users that "commits can't be snapshots", because
it would take up too much disk space to store every version of
every commit. So I find that sometimes explaining a little bit about the
implementation can make the information more memorable.

Certainly I'm not able to remember details that don't make sense
with my mental model of how computers work and I don't expect other
people to either, so I think it's important to give an explanation that
handles the biggest "objections".

> 1. That it’s a packfile and where it is might be too much detail for
>    this doc
> 2. I vaguely recall documents discussing what happens to “storing every
>    version” discussing deltas instead of packs? Again, I am not a Git
>    developer though.

I could be wrong about the details here, I'm not a Git developer either.
From https://git-scm.com/book/en/v2/Git-Internals-Packfiles
it looks like packfiles are implemented using deltas.

>> +
>> +References can either be:
>> +
>> +1. References to an object ID, usually a <<commit,commit>> ID
>> +2. References to another reference. This is called a "symbolic
>> reference".
>
> You seem to have used `**` when introducing terms:
>
>     This is a *symbolic reference*

Thanks, will take a look at that.

>> +[[reflogs]]
>> +REFLOGS
>> +-------
>> +
>> +Git stores the history of branch, tag, and HEAD refs in a reflog
>> +(you should read "reflog" as "ref log"). Not every ref is logged by
>
> You’ve heard of the re-flog too?

haha exactly, I just want folks to understand why it's called that :)

> I appreciate that this is the first version and you might have plans
> after this one. But I wonder if this doc could use a fair number of
> `gitlink` to branch out to all the other parts. Like git-reflog(1),
> gitglossary(7).

That's reasonable. Do you often use the "See also" section of
man pages? I've never looked at them so I'm always curious about
how people are actually using them in practice.

I also need to think about what else could link *to* this, because
without attention to discoverability probably nobody will find it.
My main idea so far is actually to add it to
https://git-scm.com/learn
but I wanted to send it here instead of adding it to the website
directly because I thought it could benefit from a more detailed
review.

> Thanks for starting on a whole new doc. That must take quite
> some effort.

All the work on documentation takes a lot of effort, in some
ways it's easier to write something new than to edit something
existing :)

On Mon, Oct 6, 2025 at 3:37 PM Julia Evans <julia@jvns.ca> wrote:
>
> Thanks for the review!
>
> >> 2. Don't mention that the full name of the branch `main` is
> >>    technically `refs/heads/main`. This should likely change but I
> >>    haven't worked out how to do it in a clear way yet.
> >
> > I think this is worth getting into.  This is a pretty
> > user-facing concept.
>
> I think I'll see if I can figure out a way to mention this and at the
> same time remove most of the rest of the references to the `.git`
> directory when explaining references (which you talked about
> further down), including packed refs.

A colleague will be explaining reflog for an audience tomorrow, and
decided to briefly explain refs, too—which tells me this is
much-needed.

For refs themselves, perhaps "git for-each-ref" is a reasonable place
to start? Since it tells you the refs you have and how to spell them
explicitly regardless of how they are stored?

-- 
D. Ben Knoble

On Mon, Oct 6, 2025, at 5:44 PM, D. Ben Knoble wrote:
> On Mon, Oct 6, 2025 at 3:37 PM Julia Evans <julia@jvns.ca> wrote:
>>
>> Thanks for the review!
>>
>> >> 2. Don't mention that the full name of the branch `main` is
>> >>    technically `refs/heads/main`. This should likely change but I
>> >>    haven't worked out how to do it in a clear way yet.
>> >
>> > I think this is worth getting into.  This is a pretty
>> > user-facing concept.
>>
>> I think I'll see if I can figure out a way to mention this and at the
>> same time remove most of the rest of the references to the `.git`
>> directory when explaining references (which you talked about
>> further down), including packed refs.
>
> A colleague will be explaining reflog for an audience tomorrow, and
> decided to briefly explain refs, too—which tells me this is
> much-needed.
>
> For refs themselves, perhaps "git for-each-ref" is a reasonable place
> to start? Since it tells you the refs you have and how to spell them
> explicitly regardless of how they are stored?

Interesting, do you use git for-each-ref? 
What do you use it for?

> -- 
> D. Ben Knoble

On Mon, Oct 6, 2025 at 5:47 PM Julia Evans <julia@jvns.ca> wrote:
>
>
>
> On Mon, Oct 6, 2025, at 5:44 PM, D. Ben Knoble wrote:
> > On Mon, Oct 6, 2025 at 3:37 PM Julia Evans <julia@jvns.ca> wrote:
> >>
> >> Thanks for the review!
> >>
> >> >> 2. Don't mention that the full name of the branch `main` is
> >> >>    technically `refs/heads/main`. This should likely change but I
> >> >>    haven't worked out how to do it in a clear way yet.
> >> >
> >> > I think this is worth getting into.  This is a pretty
> >> > user-facing concept.
> >>
> >> I think I'll see if I can figure out a way to mention this and at the
> >> same time remove most of the rest of the references to the `.git`
> >> directory when explaining references (which you talked about
> >> further down), including packed refs.
> >
> > A colleague will be explaining reflog for an audience tomorrow, and
> > decided to briefly explain refs, too—which tells me this is
> > much-needed.
> >
> > For refs themselves, perhaps "git for-each-ref" is a reasonable place
> > to start? Since it tells you the refs you have and how to spell them
> > explicitly regardless of how they are stored?
>
> Interesting, do you use git for-each-ref?
> What do you use it for?

Ah, yes, but primarily for scripting.

What I should have clarified is that "the tool (I know of) to
interrogate the refs you currently have is git-for-each-ref" (like how
git-ls-remote is the tool to interrogate a remote's refs). It avoids
the issues with assuming "tree .git/refs" or similar will capture the
actual data.

-- 
D. Ben Knoble

On Mon, Oct 6, 2025, at 05:32, Junio C Hamano wrote:
> "Julia Evans via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
>> +MAN7_TXT += gitdatamodel.adoc
>>  MAN7_TXT += gitdiffcore.adoc
>> ...
>> +gitdatamodel(7)
>> +===============
>> +
>> +NAME
>> +----
>> +gitdatamodel - Git's core data model
>> +
>> +DESCRIPTION
>> +-----------
>
> The above causes doc-lint to barf.
>[snip]
> You can check locally with "make check-docs" without waiting for my
> integration cycle to push to GitHub CI.

I think you meant `make lint-docs` for both of these.

On Fri, Oct 03, 2025 at 05:34:36PM +0000, Julia Evans via GitGitGadget wrote:
> diff --git a/Documentation/gitdatamodel.adoc b/Documentation/gitdatamodel.adoc
> new file mode 100644
> index 0000000000..4b2cb167dc
> --- /dev/null
> +++ b/Documentation/gitdatamodel.adoc
> @@ -0,0 +1,226 @@
> +gitdatamodel(7)
> +===============
> +
> +NAME
> +----
> +gitdatamodel - Git's core data model
> +
> +DESCRIPTION
> +-----------
> +
> +It's not necessary to understand Git's data model to use Git, but it's
> +very helpful when reading Git's documentation so that you know what it
> +means when the documentation says "object" "reference" or "index".

There's a missing comma after "object".

> +
> +Git's core operations use 4 kinds of data:
> +
> +1. <<objects,Objects>>: commits, trees, blobs, and tag objects
> +2. <<references,References>>: branches, tags,
> +   remote-tracking branches, etc
> +3. <<index,The index>>, also known as the staging area
> +4. <<reflogs,Reflogs>>

This list makes sense to me. There's of course more data structures in
Git, but all the other data structures shouldn't really matter to users
at all as they are mostly caches or internal details of the on-disk
format.

There's potentially one exception though, namely the Git configuration.
I'd claim that Git "uses" the Git configuration similarly to how it uses
the others, but I get why it's not explicitly mentioned here.

> +[[objects]]
> +OBJECTS
> +-------
> +
> +Commits, trees, blobs, and tag objects are all stored in Git's object database.
> +Every object has:
> +
> +1. an *ID*, which is the SHA-1 hash of its contents.

I think this needs to be adapted to not single out SHA-1 as the only
hashing algorithm. We already support SHA-256, so we should definitely
say that the algorithm can be swapped. Maybe something like:

  An *object ID*, which is the cryptographic hash of its contents. By
  default, Git uses SHA-1 as object hash, but alternative hashes like
  SHA-256 are supported.

> +  It's fast to look up a Git object using its ID.
> +  The ID is usually represented in hexadecimal, like
> +  `1b61de420a21a2f1aaef93e38ecd0e45e8bc9f0a`.
> +2. a *type*. There are 4 types of objects:
> +   <<commit,commits>>, <<tree,trees>>, <<blob,blobs>>,
> +   and <<tag-object,tag objects>>.
> +3. *contents*. The structure of the contents depends on the type.

Nit: every object also has an object size. Not sure though whether it's
fine to imply that with "contents".

> +Once an object is created, it can never be changed.
> +Here are the 4 types of objects:
> +
> +[[commit]]
> +commits::
> +    A commit contains:
> ++
> +1. Its *parent commit ID(s)*. The first commit in a repository has 0 parents,
> +  regular commits have 1 parent, merge commits have 2+ parents

I'd say "at least two parents" instead of "2+ parents".

> +2. A *commit message*
> +3. All the *files* in the commit, stored as a *<<tree,tree>>*
> +4. An *author* and the time the commit was authored
> +5. A *committer* and the time the commit was committed
> ++
> +Here's how an example commit is stored:
> ++
> +----
> +tree 1b61de420a21a2f1aaef93e38ecd0e45e8bc9f0a
> +parent 4ccb6d7b8869a86aae2e84c56523f8705b50c647
> +author Maya <maya@example.com> 1759173425 -0400
> +committer Maya <maya@example.com> 1759173425 -0400
> +
> +Add README
> +----

In practice, commits can have other headers that are ignored by Git. But
that's certainly not part of Git's core data model, so I don't think we
should mention that here.

> +Like all other objects, commits can never be changed after they're created.
> +For example, "amending" a commit with `git commit --amend` creates a new commit.
> +The old commit will eventually be deleted by `git gc`.

If we mention git-gc(1) I think it would make sense to use
`linkgit:git-gc[1]` instead to provide a link to its man page.

> +[[tree]]
> +trees::
> +    A tree is how Git represents a directory. It lists, for each item in
> +    the tree:
> ++
> +1. The *permissions*, for example `100644`

I think we should rather call these "mode bits". These bits are
permissions indeed when you have a blob, but for subtrees, symlinks and
submodules they aren't.

> +2. The *type*: either <<blob,`blob`>> (a file), `tree` (a directory),
> +  or <<commit,`commit`>> (a Git submodule)

There's also symlinks.

> +3. The *object ID*
> +4. The *filename*
> ++
> +For example, this is how a tree containing one directory (`src`) and one file
> +(`README.md`) is stored:
> ++
> +----
> +100644 blob 8728a858d9d21a8c78488c8b4e70e531b659141f README.md
> +040000 tree 89b1d2e0495f66d6929f4ff76ff1bb07fc41947d src
> +----
> ++
> +*NOTE:* The permissions are in the same format as UNIX permissions, but
> +the only allowed permissions for files (blobs) are 644 and 755.
> +
> +[[blob]]
> +blobs::
> +    A blob is how Git represents a file. A blob object contains the
> +    file's contents.
> ++
> +Storing a new blob for every new version of a file can get big, so
> +`git gc` periodically compresses objects for efficiency in `.git/objects/pack`.

I would claim that it's not necessary to mention object compression.
This should be a low-level detail that users don't ever have to worry
about. Furthermore, packing objects isn't only relevant in the context
of blobs: trees for example also tend to compress very well as there
typically is only small incremental updates to trees.

> +[[tag-object]]
> +tag objects::
> +    Tag objects (also known as "annotated tags") contain:
> ++
> +1. The *tagger* and tag date
> +2. A *tag message*, similar to a commit message
> +3. The *ID* of the object (often a commit) that they reference

They can also be signed, if we want to mention that.

> +[[references]]
> +REFERENCES
> +----------
> +
> +References are a way to give a name to a commit.
> +It's easier to remember "the changes I'm working on are on the `turtle`
> +branch" than "the changes are in commit bb69721404348e".
> +Git often uses "ref" as shorthand for "reference".
> +
> +References that you create are stored in the `.git/refs` directory,
> +and Git has a few special internal references like `HEAD` that are stored
> +in the base `.git` directory.

This isn't true anymore with the introduction of the reftable backend,
which is slated to become the default backend. I'd argue that this is
another implementation detail that the user shouldn't have to worry
about.

> +References can either be:
> +
> +1. References to an object ID, usually a <<commit,commit>> ID
> +2. References to another reference. This is called a "symbolic reference".
> +
> +Git handles references differently based on which subdirectory of
> +`.git/refs` they're stored in.

So instead of saying "subdirectory", I'd rather say "reference
hierarchy".

In general, I think we should explain that references are layed out
in a hierarchy. This is somewhat obvious with the "files" backend, as we
use directories there. But as we move on to the "reftable" backend this
may become less obvious over time.

> +Here are the main types:
> +
> +[[branch]]
> +branches: `.git/refs/heads/<name>`::

Here and in the other cases we should then strip the `.git/` prefix.

> +    A branch is a name for a commit ID.
> +    That commit is the latest commit on the branch.
> +    Branches are stored in the `.git/refs/heads/` directory.
> ++
> +To get the history of commits on a branch, Git will start at the commit
> +ID the branch references, and then look at the commit's parent(s),
> +the parent's parent, etc.
> +
> +[[tag]]
> +tags: `.git/refs/tags/<name>`::
> +    A tag is a name for a commit ID, tag object ID, or other object ID.
> +    Tags are stored in the `refs/tags/` directory.
> ++
> +Even though branches and commits are both "a name for a commit ID", Git
> +treats them very differently.
> +Branches are expected to be regularly updated as you work on the branch,
> +but it's expected that a tag will never change after you create it.

This sounds a bit like the user itself needs to update the branch. How
about this instead:

    Even though branches and commits are both "a name for a commit ID", Git
    treats them very differently:

        - Branches can be checked out directly. If so, creating a new
          commit will automatically update the checked-out branch to
          point to the new commit.

        - Tags cannot be checked out directly and don't move when
          creating a new commit. Instead, one can only check out the
          commit that a branch points to. This is called "detached
          HEAD", and the effect is that a new commit will not update 

> +[[HEAD]]
> +HEAD: `.git/HEAD`::
> +    `HEAD` is where Git stores your current <<branch,branch>>.
> +    `HEAD` is normally a symbolic reference to your current branch, for
> +    example `ref: refs/heads/main` if your current branch is `main`.
> +    `HEAD` can also be a direct reference to a commit ID,
> +    that's called "detached HEAD state".
> +
> +[[remote-tracking-branch]]
> +remote tracking branches: `.git/refs/remotes/<remote>/<branch>`::
> +    A remote-tracking branch is a name for a commit ID.
> +    It's how Git stores the last-known state of a branch in a remote
> +    repository. `git fetch` updates remote-tracking branches. When
> +    `git status` says "you're up to date with origin/main", it's looking at
> +    this.

This misses "refs/remotes/<remote>/HEAD". This reference is a symbolic
reference that indicates the default branch on the remote side.

> +[[other-refs]]
> +Other references::
> +    Git tools may create references in any subdirectory of `.git/refs`.
> +    For example, linkgit:git-stash[1], linkgit:git-bisect[1],
> +    and linkgit:git-notes[1] all create their own references
> +    in `.git/refs/stash`, `.git/refs/bisect`, etc.
> +    Third-party Git tools may also create their own references.
> ++
> +Git may also create references in the base `.git` directory
> +other than `HEAD`, like `ORIG_HEAD`.

Let's mention that such references are typically spelt all-uppercase
with underscores between. You shouldn't ever create a reference that is
for example called ".git/foo".

We enforce this restriction inconsistently, only, but I don't think that
should keep us from spelling out the common rule.

> +*NOTE:* As an optimization, references may be stored as packed
> +refs instead of in `.git/refs`. See linkgit:git-pack-refs[1].

I'd drop this note. It's an internal implementation detail and only true
for the "files" backend. The "reftable" backend stores references quite
differently and doesn't really "pack" references.

> +[[index]]
> +THE INDEX
> +---------
> +
> +The index, also known as the "staging area", contains the current staged

Honestly, I always forget which of these two nouns we are supposed to
use nowadays. I think consensus was to use "index" and avoid using
"staging area"? Not sure though, but I think we should only mention
one of these.

> +version of every file in your Git repository. When you commit, the files
> +in the index are used as the files in the next commit.
> +
> +Unlike a tree, the index is a flat list of files.
> +Each index entry has 4 fields:
> +
> +1. The *permissions*
> +2. The *<<blob,blob>> ID* of the file
> +3. The *filename*
> +4. The *number*. This is normally 0, but if there's a merge conflict

I think we don't call this "number", but "stage".

> +   there can be multiple versions (with numbers 0, 1, 2, ..)
> +   of the same filename in the index.
> +
> +It's extremely uncommon to look at the index directly: normally you'd
> +run `git status` to see a list of changes between the index and <<HEAD,HEAD>>.
> +But you can use `git ls-files --stage` to see the index.
> +Here's the output of `git ls-files --stage` in a repository with 2 files:
> +
> +----
> +100644 8728a858d9d21a8c78488c8b4e70e531b659141f 0 README.md
> +100644 665c637a360874ce43bf74018768a96d2d4d219a 0 src/hello.py
> +----
> +
> +[[reflogs]]
> +REFLOGS
> +-------
> +
> +Git stores the history of branch, tag, and HEAD refs in a reflog
> +(you should read "reflog" as "ref log"). Not every ref is logged by
> +default, but any ref can be logged.

If we mention this here, do we maybe want to mention how the user can
decide which references are logged?

> +Each reflog entry has:
> +
> +1. *Before/after *commit IDs*

This will probably misformat as we have three asterisks here, not two.

> +2. *User* who made the change, for example `Maya <maya@example.com>`
> +3. *Timestamp*

Suggestion: "*Timestamp* when that change has been made".

> +4. *Log message*, for example `pull: Fast-forward`
> +
> +Reflogs only log changes made in your local repository.
> +They are not shared with remotes.

We may want ot mention that you can reference reflog entries via
`refs/heads/<branch>@{<reflog-nr>}`.

In general, one thing that I think would be important to highlight in
this document is revisions. Most of the commands tend to not accept
references, but revisions instead, which are a lot more flexible. They
use our do-what-I-mean mechanism to resolve, but also allow the user to
specify commits relative to one another. It's probably sufficient though
to mention them briefly and then redirect to girevisions(7).

Thanks for working on this!

Patrick

"Kristoffer Haugsbakk" <kristofferhaugsbakk@fastmail.com> writes:

> On Mon, Oct 6, 2025, at 05:32, Junio C Hamano wrote:
>> "Julia Evans via GitGitGadget" <gitgitgadget@gmail.com> writes:
>>
>>> +MAN7_TXT += gitdatamodel.adoc
>>>  MAN7_TXT += gitdiffcore.adoc
>>> ...
>>> +gitdatamodel(7)
>>> +===============
>>> +
>>> +NAME
>>> +----
>>> +gitdatamodel - Git's core data model
>>> +
>>> +DESCRIPTION
>>> +-----------
>>
>> The above causes doc-lint to barf.
>>[snip]
>> You can check locally with "make check-docs" without waiting for my
>> integration cycle to push to GitHub CI.
>
> I think you meant `make lint-docs` for both of these.

The former is a typo for "causes lint-docs to barf", but I did mean
"make check-docs" as the recipe for local checking.

You could also do "make -C Documentation lint-docs", but that is a
lot more to type ;-).

Thanks.

Patrick Steinhardt <ps@pks.im> writes:

>> +Git's core operations use 4 kinds of data:
>> +
>> +1. <<objects,Objects>>: commits, trees, blobs, and tag objects
>> +2. <<references,References>>: branches, tags,
>> +   remote-tracking branches, etc
>> +3. <<index,The index>>, also known as the staging area
>> +4. <<reflogs,Reflogs>>
>
> This list makes sense to me. There's of course more data structures in
> Git, but all the other data structures shouldn't really matter to users
> at all as they are mostly caches or internal details of the on-disk
> format.
>
> There's potentially one exception though, namely the Git configuration.
> I'd claim that Git "uses" the Git configuration similarly to how it uses
> the others, but I get why it's not explicitly mentioned here.

The core operations do not use Git configuration any more than they
use what is specified by the command line arguments.

>> +[[objects]]
>> +OBJECTS
>> +-------
>> +
>> +Commits, trees, blobs, and tag objects are all stored in Git's object database.
>> +Every object has:
>> +
>> +1. an *ID*, which is the SHA-1 hash of its contents.
>
> I think this needs to be adapted to not single out SHA-1 as the only
> hashing algorithm. We already support SHA-256, so we should definitely
> say that the algorithm can be swapped. Maybe something like:

Good point.  Also officially they are called "object name".

>   An *object ID*, which is the cryptographic hash of its contents. By
>   default, Git uses SHA-1 as object hash, but alternative hashes like
>   SHA-256 are supported.

I'd avoid "object name is the result of hashing X" which historically
was a source of question: "why does 'sha1sum README.md' give different
hash from 'git add README.md && git ls-files -s README.md'?"

It is an irrelevant implementation detail (and you'd eventually end
up having to say "X is <type> SP <length> NUL <contents>").

    An object name, which is derived cryptographically from its
    type, size and contents.  All versions of Git can use SHA-1 hash
    function, but more recent versions of Git can also use SHA-256
    hash function.

>> +commits::
>> +    A commit contains:
>> ++
>> +1. Its *parent commit ID(s)*. The first commit in a repository has 0 parents,
>> +  regular commits have 1 parent, merge commits have 2+ parents
>
> I'd say "at least two parents" instead of "2+ parents".

Yup, that reads much better.

>> +tree 1b61de420a21a2f1aaef93e38ecd0e45e8bc9f0a
>> +parent 4ccb6d7b8869a86aae2e84c56523f8705b50c647
>> +author Maya <maya@example.com> 1759173425 -0400
>> +committer Maya <maya@example.com> 1759173425 -0400
>> +
>> +Add README
>> +----
>
> In practice, commits can have other headers that are ignored by Git. But
> that's certainly not part of Git's core data model, so I don't think we
> should mention that here.

Third-party software can add truly garbage ones that do not have any
meaning, and Git tolerates by ignoring them.  But there are others
that Git does pay attention to, like encoding, gpgsig, etc., which
may worth mention (in the form that "these four are what you typically
see, but there may be others" without even naming any).

On Tue, Oct 7, 2025 at 11:51 AM Patrick Steinhardt <ps@pks.im> wrote:
>
> On Fri, Oct 03, 2025 at 05:34:36PM +0000, Julia Evans via GitGitGadget wrote:
[snip]
> > +    A branch is a name for a commit ID.
> > +    That commit is the latest commit on the branch.
> > +    Branches are stored in the `.git/refs/heads/` directory.
> > ++
> > +To get the history of commits on a branch, Git will start at the commit
> > +ID the branch references, and then look at the commit's parent(s),
> > +the parent's parent, etc.
> > +
> > +[[tag]]
> > +tags: `.git/refs/tags/<name>`::
> > +    A tag is a name for a commit ID, tag object ID, or other object ID.
> > +    Tags are stored in the `refs/tags/` directory.
> > ++
> > +Even though branches and commits are both "a name for a commit ID", Git
> > +treats them very differently.
> > +Branches are expected to be regularly updated as you work on the branch,
> > +but it's expected that a tag will never change after you create it.
>
> This sounds a bit like the user itself needs to update the branch. How
> about this instead:
>
>     Even though branches and commits are both "a name for a commit ID", Git
>     treats them very differently:
>
>         - Branches can be checked out directly. If so, creating a new
>           commit will automatically update the checked-out branch to
>           point to the new commit.
>
>         - Tags cannot be checked out directly and don't move when
>           creating a new commit. Instead, one can only check out the
>           commit that a branch points to. This is called "detached
>           HEAD", and the effect is that a new commit will not update

missing "the tag." ?

On Tue, Oct 7, 2025, at 10:32 AM, Patrick Steinhardt wrote:
> On Fri, Oct 03, 2025 at 05:34:36PM +0000, Julia Evans via GitGitGadget wrote:
>> diff --git a/Documentation/gitdatamodel.adoc b/Documentation/gitdatamodel.adoc
>> new file mode 100644
>> index 0000000000..4b2cb167dc
>> --- /dev/null
>> +++ b/Documentation/gitdatamodel.adoc
>> @@ -0,0 +1,226 @@
>> +gitdatamodel(7)
>> +===============
>> +
>> +NAME
>> +----
>> +gitdatamodel - Git's core data model
>> +
>> +DESCRIPTION
>> +-----------
>> +
>> +It's not necessary to understand Git's data model to use Git, but it's
>> +very helpful when reading Git's documentation so that you know what it
>> +means when the documentation says "object" "reference" or "index".
>
> There's a missing comma after "object".

Will fix.

>> +
>> +Git's core operations use 4 kinds of data:
>> +
>> +1. <<objects,Objects>>: commits, trees, blobs, and tag objects
>> +2. <<references,References>>: branches, tags,
>> +   remote-tracking branches, etc
>> +3. <<index,The index>>, also known as the staging area
>> +4. <<reflogs,Reflogs>>
>
> This list makes sense to me. There's of course more data structures in
> Git, but all the other data structures shouldn't really matter to users
> at all as they are mostly caches or internal details of the on-disk
> format.
>
> There's potentially one exception though, namely the Git configuration.
> I'd claim that Git "uses" the Git configuration similarly to how it uses
> the others, but I get why it's not explicitly mentioned here.
>
>> +[[objects]]
>> +OBJECTS
>> +-------
>> +
>> +Commits, trees, blobs, and tag objects are all stored in Git's object database.
>> +Every object has:
>> +
>> +1. an *ID*, which is the SHA-1 hash of its contents.
>
> I think this needs to be adapted to not single out SHA-1 as the only
> hashing algorithm. We already support SHA-256, so we should definitely
> say that the algorithm can be swapped. Maybe something like:
>
>   An *object ID*, which is the cryptographic hash of its contents. By
>   default, Git uses SHA-1 as object hash, but alternative hashes like
>   SHA-256 are supported.

Makes sense. I might just say "cryptographic hash of its type and contents"
and leave it that. I'm not sure it's worth getting into details
of the exact hash function.

>> +  It's fast to look up a Git object using its ID.
>> +  The ID is usually represented in hexadecimal, like
>> +  `1b61de420a21a2f1aaef93e38ecd0e45e8bc9f0a`.
>> +2. a *type*. There are 4 types of objects:
>> +   <<commit,commits>>, <<tree,trees>>, <<blob,blobs>>,
>> +   and <<tag-object,tag objects>>.
>> +3. *contents*. The structure of the contents depends on the type.
>
> Nit: every object also has an object size. Not sure though whether it's
> fine to imply that with "contents".

I think it is.

>> +Once an object is created, it can never be changed.
>> +Here are the 4 types of objects:
>> +
>> +[[commit]]
>> +commits::
>> +    A commit contains:
>> ++
>> +1. Its *parent commit ID(s)*. The first commit in a repository has 0 parents,
>> +  regular commits have 1 parent, merge commits have 2+ parents
>
> I'd say "at least two parents" instead of "2+ parents".
>
>> +2. A *commit message*
>> +3. All the *files* in the commit, stored as a *<<tree,tree>>*
>> +4. An *author* and the time the commit was authored
>> +5. A *committer* and the time the commit was committed
>> ++
>> +Here's how an example commit is stored:
>> ++
>> +----
>> +tree 1b61de420a21a2f1aaef93e38ecd0e45e8bc9f0a
>> +parent 4ccb6d7b8869a86aae2e84c56523f8705b50c647
>> +author Maya <maya@example.com> 1759173425 -0400
>> +committer Maya <maya@example.com> 1759173425 -0400
>> +
>> +Add README
>> +----
>
> In practice, commits can have other headers that are ignored by Git. But
> that's certainly not part of Git's core data model, so I don't think we
> should mention that here.
>
>> +Like all other objects, commits can never be changed after they're created.
>> +For example, "amending" a commit with `git commit --amend` creates a new commit.
>> +The old commit will eventually be deleted by `git gc`.
>
> If we mention git-gc(1) I think it would make sense to use
> `linkgit:git-gc[1]` instead to provide a link to its man page.

Agreed.

>> +[[tree]]
>> +trees::
>> +    A tree is how Git represents a directory. It lists, for each item in
>> +    the tree:
>> ++
>> +1. The *permissions*, for example `100644`
>
> I think we should rather call these "mode bits". These bits are
> permissions indeed when you have a blob, but for subtrees, symlinks and
> submodules they aren't.

I think it's a bit strange to call them mode bits since I thought they were stored
as ASCII strings and it's basically an enum of 5 options, but I see your point.
I think "file mode" will work and that's used elsewhere.

I wonder if it would make sense to list all of the possible file modes if
this isn't documented anywhere else, my impression is that it's a short
list and that it's unlikely to change much in the future.

And listing them all might make it more clear that Git's file modes don't
have much in common with Unix file modes.
I looked for where this is documented and it looks like the only place is
in `man git-fast-import` . That man page says that there are just 5 options
(040000, 160000, 100644, 100755, 120000)

>> +2. The *type*: either <<blob,`blob`>> (a file), `tree` (a directory),
>> +  or <<commit,`commit`>> (a Git submodule)
>
> There's also symlinks.

I created a test symlink and it looks like symlinks are stored as type "blob".
I might say which type corresponds to which file mode,
though I'm not sure what type corresponds to the "gitlink" mode (commit?).

I think these are the 5 modes and what they mean / what type they
should have. Not sure about the gitlink mode though.

  - `100644`: regular file (with type `blob`)
  - `100755`: executable file (with type `blob`)
  - `120000`: symbolic link (with type `blob`)
  - `040000`: directory (with type `tree`)
  - `160000`: gitlink, for use with submodules (with type `commit`)

>> +3. The *object ID*
>> +4. The *filename*
>> ++
>> +For example, this is how a tree containing one directory (`src`) and one file
>> +(`README.md`) is stored:
>> ++
>> +----
>> +100644 blob 8728a858d9d21a8c78488c8b4e70e531b659141f README.md
>> +040000 tree 89b1d2e0495f66d6929f4ff76ff1bb07fc41947d src
>> +----
>> ++
>> +*NOTE:* The permissions are in the same format as UNIX permissions, but
>> +the only allowed permissions for files (blobs) are 644 and 755.
>> +
>> +[[blob]]
>> +blobs::
>> +    A blob is how Git represents a file. A blob object contains the
>> +    file's contents.
>> ++
>> +Storing a new blob for every new version of a file can get big, so
>> +`git gc` periodically compresses objects for efficiency in `.git/objects/pack`.
>
> I would claim that it's not necessary to mention object compression.
> This should be a low-level detail that users don't ever have to worry
> about. Furthermore, packing objects isn't only relevant in the context
> of blobs: trees for example also tend to compress very well as there
> typically is only small incremental updates to trees.

I discussed why I think this important in another reply,
https://lore.kernel.org/all/51e0a55c-1f1d-4cae-9459-8c2b9220e52d@app.fastmail.com/,
will paste what I said here. I'll think about this more though.

paste follows:

That's true! The reason I think this is important to mention is that I find
that people often "reject" information that they find implausible, even
if it comes from a credible source. ("that can't be true! I must be
not understanding correctly. Oh well, I'll just ignore that!")

I sometimes hear from users that "commits can't be snapshots", because
it would take up too much disk space to store every version of
every commit. So I find that sometimes explaining a little bit about the
implementation can make the information more memorable.

Certainly I'm not able to remember details that don't make sense
with my mental model of how computers work and I don't expect other
people to either, so I think it's important to give an explanation that
handles the biggest "objections".

>> +[[tag-object]]
>> +tag objects::
>> +    Tag objects (also known as "annotated tags") contain:
>> ++
>> +1. The *tagger* and tag date
>> +2. A *tag message*, similar to a commit message
>> +3. The *ID* of the object (often a commit) that they reference
>
> They can also be signed, if we want to mention that.

I guess that's true for commit objects too. Not sure whether to
mention it either, can add it if others think it's important.

>> +[[references]]
>> +REFERENCES
>> +----------
>> +
>> +References are a way to give a name to a commit.
>> +It's easier to remember "the changes I'm working on are on the `turtle`
>> +branch" than "the changes are in commit bb69721404348e".
>> +Git often uses "ref" as shorthand for "reference".
>> +
>> +References that you create are stored in the `.git/refs` directory,
>> +and Git has a few special internal references like `HEAD` that are stored
>> +in the base `.git` directory.
>
> This isn't true anymore with the introduction of the reftable backend,
> which is slated to become the default backend. I'd argue that this is
> another implementation detail that the user shouldn't have to worry
> about.

Makes sense, will fix. (as well as other references to the .git prefix and
"subdirectories").

>> +References can either be:
>> +
>> +1. References to an object ID, usually a <<commit,commit>> ID
>> +2. References to another reference. This is called a "symbolic reference".
>> +
>> +Git handles references differently based on which subdirectory of
>> +`.git/refs` they're stored in.
>
> So instead of saying "subdirectory", I'd rather say "reference
> hierarchy".
>
> In general, I think we should explain that references are layed out
> in a hierarchy. This is somewhat obvious with the "files" backend, as we
> use directories there. But as we move on to the "reftable" backend this
> may become less obvious over time.

That makes sense.

>> +[[tag]]
>> +tags: `.git/refs/tags/<name>`::
>> +    A tag is a name for a commit ID, tag object ID, or other object ID.
>> +    Tags are stored in the `refs/tags/` directory.
>> ++
>> +Even though branches and commits are both "a name for a commit ID", Git
>> +treats them very differently.
>> +Branches are expected to be regularly updated as you work on the branch,
>> +but it's expected that a tag will never change after you create it.
>
> This sounds a bit like the user itself needs to update the branch. How
> about this instead:
>
>     Even though branches and commits are both "a name for a commit ID", Git
>     treats them very differently:
>
>         - Branches can be checked out directly. If so, creating a new
>           commit will automatically update the checked-out branch to
>           point to the new commit.
>
>         - Tags cannot be checked out directly and don't move when
>           creating a new commit. Instead, one can only check out the
>           commit that a branch points to. This is called "detached
>           HEAD", and the effect is that a new commit will not update 

I think mentioning that branches can be checked out and that tags can't
is a good idea.

>> +[[HEAD]]
>> +HEAD: `.git/HEAD`::
>> +    `HEAD` is where Git stores your current <<branch,branch>>.
>> +    `HEAD` is normally a symbolic reference to your current branch, for
>> +    example `ref: refs/heads/main` if your current branch is `main`.
>> +    `HEAD` can also be a direct reference to a commit ID,
>> +    that's called "detached HEAD state".
>> +
>> +[[remote-tracking-branch]]
>> +remote tracking branches: `.git/refs/remotes/<remote>/<branch>`::
>> +    A remote-tracking branch is a name for a commit ID.
>> +    It's how Git stores the last-known state of a branch in a remote
>> +    repository. `git fetch` updates remote-tracking branches. When
>> +    `git status` says "you're up to date with origin/main", it's looking at
>> +    this.
>
> This misses "refs/remotes/<remote>/HEAD". This reference is a symbolic
> reference that indicates the default branch on the remote side.

Is "refs/remotes/<remote>/HEAD" a remote-tracking branch?
I've never thought about that reference and I'm not sure what to call it.

>> +[[other-refs]]
>> +Other references::
>> +    Git tools may create references in any subdirectory of `.git/refs`.
>> +    For example, linkgit:git-stash[1], linkgit:git-bisect[1],
>> +    and linkgit:git-notes[1] all create their own references
>> +    in `.git/refs/stash`, `.git/refs/bisect`, etc.
>> +    Third-party Git tools may also create their own references.
>> ++
>> +Git may also create references in the base `.git` directory
>> +other than `HEAD`, like `ORIG_HEAD`.
>
> Let's mention that such references are typically spelt all-uppercase
> with underscores between. You shouldn't ever create a reference that is
> for example called ".git/foo".
>
> We enforce this restriction inconsistently, only, but I don't think that
> should keep us from spelling out the common rule.

That makes sense. I'm also not sure whether third-party
Git tools are "supposed" to create references outside of "refs/",
or whether that's common. 

>> +*NOTE:* As an optimization, references may be stored as packed
>> +refs instead of in `.git/refs`. See linkgit:git-pack-refs[1].
>
> I'd drop this note. It's an internal implementation detail and only true
> for the "files" backend. The "reftable" backend stores references quite
> differently and doesn't really "pack" references.
>
>> +[[index]]
>> +THE INDEX
>> +---------
>> +
>> +The index, also known as the "staging area", contains the current staged
>
> Honestly, I always forget which of these two nouns we are supposed to
> use nowadays. I think consensus was to use "index" and avoid using
> "staging area"? Not sure though, but I think we should only mention
> one of these.
>
>> +version of every file in your Git repository. When you commit, the files
>> +in the index are used as the files in the next commit.
>> +
>> +Unlike a tree, the index is a flat list of files.
>> +Each index entry has 4 fields:
>> +
>> +1. The *permissions*
>> +2. The *<<blob,blob>> ID* of the file
>> +3. The *filename*
>> +4. The *number*. This is normally 0, but if there's a merge conflict
>
> I think we don't call this "number", but "stage".

Thanks, I see that it's sometimes called "stage number" which is a little
easier to search for so I'll call it that.

>> +   there can be multiple versions (with numbers 0, 1, 2, ..)
>> +   of the same filename in the index.
>> +
>> +It's extremely uncommon to look at the index directly: normally you'd
>> +run `git status` to see a list of changes between the index and <<HEAD,HEAD>>.
>> +But you can use `git ls-files --stage` to see the index.
>> +Here's the output of `git ls-files --stage` in a repository with 2 files:
>> +
>> +----
>> +100644 8728a858d9d21a8c78488c8b4e70e531b659141f 0 README.md
>> +100644 665c637a360874ce43bf74018768a96d2d4d219a 0 src/hello.py
>> +----
>> +
>> +[[reflogs]]
>> +REFLOGS
>> +-------
>> +
>> +Git stores the history of branch, tag, and HEAD refs in a reflog
>> +(you should read "reflog" as "ref log"). Not every ref is logged by
>> +default, but any ref can be logged.
>
> If we mention this here, do we maybe want to mention how the user can
> decide which references are logged?

Do you mean by using the setting `core.logAllRefUpdates`?

>> +Each reflog entry has:
>> +
>> +1. *Before/after *commit IDs*
>
> This will probably misformat as we have three asterisks here, not two.
>
>> +2. *User* who made the change, for example `Maya <maya@example.com>`
>> +3. *Timestamp*
>
> Suggestion: "*Timestamp* when that change has been made".

Makes sense.

>> +4. *Log message*, for example `pull: Fast-forward`
>> +
>> +Reflogs only log changes made in your local repository.
>> +They are not shared with remotes.
>
> We may want ot mention that you can reference reflog entries via
> `refs/heads/<branch>@{<reflog-nr>}`.
>
> In general, one thing that I think would be important to highlight in
> this document is revisions. Most of the commands tend to not accept
> references, but revisions instead, which are a lot more flexible. They
> use our do-what-I-mean mechanism to resolve, but also allow the user to
> specify commits relative to one another. It's probably sufficient though
> to mention them briefly and then redirect to girevisions(7).

Will think about this, I'm not sure how to best incorporate that.
Maybe under the commits section.

> Thanks for working on this!

Thanks for the review!

- Julia

"Julia Evans via GitGitGadget" <gitgitgadget@gmail.com> writes:

> diff --git a/Documentation/gitdatamodel.adoc b/Documentation/gitdatamodel.adoc
> new file mode 100644
> index 0000000000..e36e833f66
> --- /dev/null
> +++ b/Documentation/gitdatamodel.adoc
> @@ -0,0 +1,286 @@
> +gitdatamodel(7)
> +===============
> +
> +NAME
> +----
> +gitdatamodel - Git's core data model
> +
> +SYNOPSIS
> +--------
> +gitdatamodel
> +
> +DESCRIPTION
> +-----------
> +
> +It's not necessary to understand Git's data model to use Git, but it's
> +very helpful when reading Git's documentation so that you know what it
> +means when the documentation says "object", "reference" or "index".

"While it is not necessary ..., it is helpful ..." may flow better
than "It is not necesary ..., but it is very helpful".

> +This means that if you have an object's ID, you can always recover its
> +exact contents as long as the object hasn't been deleted.

Somewhere in distant footnote, we may want to mention that objects
that are in use are never deleted, and when they get removed (i.e.,
garbage collection).  As part of the data model, "everything is
retained by default, until we can prove it is no longer reachable"
probably belongs somewhere.

> +Here's how each type of object is structured:
> +
> +[[commit]]
> +commit::
> +    A commit contains the full directory structure of every file
> +    in that version of the repository and each file's contents.

What you are describing here is more of the property of a tree; a
commit is a bit richer.

    A commit records a snapshot of the every file in the project at
    one point in time, records who contributed to create such a
    snapshot and why, and how that particular snapshot relates to
    other snapshots in the history.

> +    It has these these required fields

"these these".

> +Like all other objects, commits can never be changed after they're created.
> +For example, "amending" a commit with `git commit --amend` creates a new
> +commit with the same parent.

"same parent." -> "same parent, without modifying the original
commit object at all"?  Maybe redundant?  I dunno.

> +[[tree]]
> +tree::
> +    A tree is how Git represents a directory.

"a directory" -> "contents in a directory"?  I dunno.

> +    It can contain files or other trees (which are subdirectories).
> +    It lists, for each item in the tree:
> ++
> +1. The *filename*, for example `hello.py`
> +2. The *type*: either <<blob,`blob`>> (a file), `tree` (a directory),
> +  or <<commit,`commit`>> (a Git submodule, which is a
> +  commit from a different Git repository)

This is a bit of white lie.  A tree object entry never stores the
type of the object.  It records <mode, object name, path component>.

The second field you see in git ls-tree output is computed from the
object name (when the object is available) or inferred from the mode
bits.

> +3. The *file mode*. Git has these file modes. which are only
> +   spiritually related to Unix permissions:

In the cover letter part of the message I am responding to, I saw
repeated mention of "permissions should be "file mode"; let's be
consistent.

"Git has these file modes, which are ..." -> 

    Git uses the following file mode to represent what each tree
    entry is (because an object of the same type, e.g. "blob", is
    used to represent more than one kind of things).  The file mode
    are assigned to resemble Unix file mode.

    Note that Git does not _store_ permissions, and there are only
    two kinds of regular files; non-executable (100644) or
    executable (100755).  To Git, there are no files that are
    "readable only by the owner" etc., so file mode bits like
    100600, 100400, etc., are never used.

> +[[tag-object]]
> +tag object::
> +    Tag objects contain these required fields
> +    (though there are other optional fields):
> ++
> +1. The *ID* and *type* of the object (often a commit) that they reference

Not wrong per-se, but it is a bit curious to lump these two into a
single enumerated item here, unlike "author" and "committer" were
enumerated separately for commit objects.  If you are going to show
"cat-file -p" output for illustration, it may be help readers
understand them if you had them separately listed here.

> +2. The *tagger* and tag date
> +3. A *tag message*, similar to a commit message

> +[[index]]
> +THE INDEX
> +---------
> +The index, also known as the "staging area", is a list of files and
> +the contents of each file, stored as a <<blob,blob>>.
> +You can add files to the index or update the contents of a file in the
> +index with linkgit:git-add[1]. This is called "staging" the file for commit.
> +
> +Unlike a <<tree,tree>>, the index is a flat list of files.

This is a bit of white lie, as modern versions of Git could be
collapsing uninteresting parts of the directory structure as a
single tree in an index entry (this is called "sparse index"), and
can expand such collapsed "tree" in the index on-demand into its
constituent files and directories.  But I do not mind presenting the
traditional world model for conceptual simplicity.

> +When you commit, Git converts the list of files in the index to a
> +directory <<tree,tree>> and uses that tree in the new <<commit,commit>>.
> +
> +Each index entry has 4 fields:
> +
> +1. The *<<tree,file mode>>*
> +2. The *<<blob,blob>> ID* of the file

If you were to collapse descriptions like you did for tag objects
where ID and TYPE were treated as a unit, here is the place to do
so.  With the mode bits and object ID, we can represent regular
files that are non-executable, regular files that are executable,  
symbolic links, and submodules (if a sparse-index is in use, an
index entry could be a subdirectory, but I suggested above that we
can ignore them for simplicity).

But <<blob,blob>> is highly misleading.  Even if we ignore
sparse-index, we may see a commit object there.

    Each index entry records

    1. The object that occupies the path, as (file mode, object
       name) tuple.  Most often, it is a regular file whose contents
       are stored in a blob object, that is either non-executable
       (100644), executable (100755), or a symbolic link (120000),
       but the object can be a commit in another repository if it
       represents a submodule.

    2. The stage number, which is normally 0, but entries with
       higher stages for the same path are used during a conflicted
       merge.

    3. The path name for the index entry.

> +3. The *file path*, for example `src/hello.py`
> +4. The *stage number*, either 0, 1, 2, or 3. This is normally 0, but if
> +   there's a merge conflict there can be multiple versions of the same
> +   filename in the index.

If you are going by "ls-files -s" output, it may be better to swap 3
and 4 above for ease of understanding.

> +It's extremely uncommon to look at the index directly: normally you'd
> +run `git status` to see a list of changes between the index and <<HEAD,HEAD>>.
> +But you can use `git ls-files --stage` to see the index.
> +Here's the output of `git ls-files --stage` in a repository with 2 files:
> +
> +----
> +100644 8728a858d9d21a8c78488c8b4e70e531b659141f 0 README.md
> +100644 665c637a360874ce43bf74018768a96d2d4d219a 0 src/hello.py
> +----
> +
> +[[reflogs]]
> +REFLOGS
> +-------
> +
> +Every time a branch, remote-tracking branch, or HEAD is updated, Git
> +updates a log called a "reflog" for that <<references,reference>>.

If we want to avoid using word X while explaining X, then we can
rephrase it as "Git updates a record in the reflog for that
reference".

>> +
>> +It's not necessary to understand Git's data model to use Git, but it's
>> +very helpful when reading Git's documentation so that you know what it
>> +means when the documentation says "object", "reference" or "index".
>
> "While it is not necessary ..., it is helpful ..." may flow better
> than "It is not necesary ..., but it is very helpful".
>
>> +This means that if you have an object's ID, you can always recover its
>> +exact contents as long as the object hasn't been deleted.
>
> Somewhere in distant footnote, we may want to mention that objects
> that are in use are never deleted, and when they get removed (i.e.,
> garbage collection).  As part of the data model, "everything is
> retained by default, until we can prove it is no longer reachable"
> probably belongs somewhere.

Agreed, I really like this idea. Came up with the following, which I'll put at
the bottom of the "References" section if I don't come up with a better idea.
(I don't feel strongly about where exactly it should go):

NOTE: Objects will only be deleted if they aren't "reachable" from any reference.
An object is "reachable" if we can find it by following tags to whatever
they tag, commits to their parents or trees, and trees to the trees or
blobs that they contain.
For example, if you amend a commit, with `git commit --amend`,
the old commit will usually not be reachable, so it may be deleted eventually.

>> +Here's how each type of object is structured:
>> +
>> +[[commit]]
>> +commit::
>> +    A commit contains the full directory structure of every file
>> +    in that version of the repository and each file's contents.
>
> What you are describing here is more of the property of a tree; a
> commit is a bit richer.
>
>     A commit records a snapshot of the every file in the project at
>     one point in time, records who contributed to create such a
>     snapshot and why, and how that particular snapshot relates to
>     other snapshots in the history.

I don't understand the goal of explaining a commit in detail in
paragraph form when we already explain everything in a commit right
below this.

My goal of this intro sentence is just to emphasize what I think is the
least obvious point in that list, which is that commits contain every file. 

Happy to change it to something shorter like
"A commit records a snapshot of the every file in the project" if you
prefer that wording.

>> +    It has these these required fields
>
> "these these".

Oops, will fix

>> +Like all other objects, commits can never be changed after they're created.
>> +For example, "amending" a commit with `git commit --amend` creates a new
>> +commit with the same parent.
>
> "same parent." -> "same parent, without modifying the original
> commit object at all"?  Maybe redundant?  I dunno.
>
>> +[[tree]]
>> +tree::
>> +    A tree is how Git represents a directory.
>
> "a directory" -> "contents in a directory"?  I dunno.
>
>> +    It can contain files or other trees (which are subdirectories).
>> +    It lists, for each item in the tree:
>> ++
>> +1. The *filename*, for example `hello.py`
>> +2. The *type*: either <<blob,`blob`>> (a file), `tree` (a directory),
>> +  or <<commit,`commit`>> (a Git submodule, which is a
>> +  commit from a different Git repository)
>
> This is a bit of white lie.  A tree object entry never stores the
> type of the object.  It records <mode, object name, path component>.
>
> The second field you see in git ls-tree output is computed from the
> object name (when the object is available) or inferred from the mode
> bits.

Thanks, I didn't realize how tree object entries were stored.
Will remove "type".

>> +3. The *file mode*. Git has these file modes. which are only
>> +   spiritually related to Unix permissions:
>
> In the cover letter part of the message I am responding to, I saw
> repeated mention of "permissions should be "file mode"; let's be
> consistent.
>
> "Git has these file modes, which are ..." -> 

Makes sense. Will change to "Unix file modes" from "Unix permissions".
I don't think this needs a more dramatic rewrite though.

>     Git uses the following file mode to represent what each tree
>     entry is (because an object of the same type, e.g. "blob", is
>     used to represent more than one kind of things).  The file mode
>     are assigned to resemble Unix file mode.
>
>     Note that Git does not _store_ permissions, and there are only
>     two kinds of regular files; non-executable (100644) or
>     executable (100755).  To Git, there are no files that are
>     "readable only by the owner" etc., so file mode bits like
>     100600, 100400, etc., are never used.
>
>> +[[tag-object]]
>> +tag object::
>> +    Tag objects contain these required fields
>> +    (though there are other optional fields):
>> ++
>> +1. The *ID* and *type* of the object (often a commit) that they reference
>
> Not wrong per-se, but it is a bit curious to lump these two into a
> single enumerated item here, unlike "author" and "committer" were
> enumerated separately for commit objects.  If you are going to show
> "cat-file -p" output for illustration, it may be help readers
> understand them if you had them separately listed here.

Agreed, I'll split them into two items.

>> +2. The *tagger* and tag date
>> +3. A *tag message*, similar to a commit message
>
>> +[[index]]
>> +THE INDEX
>> +---------
>> +The index, also known as the "staging area", is a list of files and
>> +the contents of each file, stored as a <<blob,blob>>.
>> +You can add files to the index or update the contents of a file in the
>> +index with linkgit:git-add[1]. This is called "staging" the file for commit.
>> +
>> +Unlike a <<tree,tree>>, the index is a flat list of files.
>
> This is a bit of white lie, as modern versions of Git could be
> collapsing uninteresting parts of the directory structure as a
> single tree in an index entry (this is called "sparse index"), and
> can expand such collapsed "tree" in the index on-demand into its
> constituent files and directories.  But I do not mind presenting the
> traditional world model for conceptual simplicity.

I didn't know that, thanks. I guess I'll leave it the way it is for now.
It could be good to add a footnote, but I don't actually know how
to add footnotes in this document format.

>> +When you commit, Git converts the list of files in the index to a
>> +directory <<tree,tree>> and uses that tree in the new <<commit,commit>>.
>> +
>> +Each index entry has 4 fields:
>> +
>> +1. The *<<tree,file mode>>*
>> +2. The *<<blob,blob>> ID* of the file
>
> If you were to collapse descriptions like you did for tag objects
> where ID and TYPE were treated as a unit, here is the place to do
> so.  With the mode bits and object ID, we can represent regular
> files that are non-executable, regular files that are executable,  
> symbolic links, and submodules (if a sparse-index is in use, an
> index entry could be a subdirectory, but I suggested above that we
> can ignore them for simplicity).
>
> But <<blob,blob>> is highly misleading.  Even if we ignore
> sparse-index, we may see a commit object there.

Thanks, I didn't realize that. Will change to say that it can be a blob
or commit ID. I don't think that collapsing will help, IMO it's
important to keep a consistent format.

>     Each index entry records
>
>     1. The object that occupies the path, as (file mode, object
>        name) tuple.  Most often, it is a regular file whose contents
>        are stored in a blob object, that is either non-executable
>        (100644), executable (100755), or a symbolic link (120000),
>        but the object can be a commit in another repository if it
>        represents a submodule.
>
>     2. The stage number, which is normally 0, but entries with
>        higher stages for the same path are used during a conflicted
>        merge.
>
>     3. The path name for the index entry.
>
>> +3. The *file path*, for example `src/hello.py`
>> +4. The *stage number*, either 0, 1, 2, or 3. This is normally 0, but if
>> +   there's a merge conflict there can be multiple versions of the same
>> +   filename in the index.
>
> If you are going by "ls-files -s" output, it may be better to swap 3
> and 4 above for ease of understanding.

Good point, will do.

>> +It's extremely uncommon to look at the index directly: normally you'd
>> +run `git status` to see a list of changes between the index and <<HEAD,HEAD>>.
>> +But you can use `git ls-files --stage` to see the index.
>> +Here's the output of `git ls-files --stage` in a repository with 2 files:
>> +
>> +----
>> +100644 8728a858d9d21a8c78488c8b4e70e531b659141f 0 README.md
>> +100644 665c637a360874ce43bf74018768a96d2d4d219a 0 src/hello.py
>> +----
>> +
>> +[[reflogs]]
>> +REFLOGS
>> +-------
>> +
>> +Every time a branch, remote-tracking branch, or HEAD is updated, Git
>> +updates a log called a "reflog" for that <<references,reference>>.
>
> If we want to avoid using word X while explaining X, then we can
> rephrase it as "Git updates a record in the reflog for that
> reference".

I think the current phrasing is okay. I also didn't respond to some of the
phrasing suggestions above if I didn't understand the goal of them.
Hope that's okay.

"Julia Evans" <julia@jvns.ca> writes:

> Agreed, I really like this idea. Came up with the following, which I'll put at
> the bottom of the "References" section if I don't come up with a better idea.
> (I don't feel strongly about where exactly it should go):
>
> NOTE: Objects will only be deleted if they aren't "reachable" from any reference.
> An object is "reachable" if we can find it by following tags to whatever
> they tag, commits to their parents or trees, and trees to the trees or
> blobs that they contain.
> For example, if you amend a commit, with `git commit --amend`,
> the old commit will usually not be reachable, so it may be deleted eventually.

Other reachability anchors exist, like the index and reflog entries,
but we have to stop at somewhere.  I am fine if we do not mention
them explicitly for the sake of simplicity.

>>> +Here's how each type of object is structured:
>>> +
>>> +[[commit]]
>>> +commit::
>>> +    A commit contains the full directory structure of every file
>>> +    in that version of the repository and each file's contents.
>>
>> What you are describing here is more of the property of a tree; a
>> commit is a bit richer.
>>
>>     A commit records a snapshot of the every file in the project at
>>     one point in time, records who contributed to create such a
>>     snapshot and why, and how that particular snapshot relates to
>>     other snapshots in the history.
>
> I don't understand the goal of explaining a commit in detail in
> paragraph form when we already explain everything in a commit right
> below this.
>
> My goal of this intro sentence is just to emphasize what I think is the
> least obvious point in that list, which is that commits contain every file. 
>
> Happy to change it to something shorter like
> "A commit records a snapshot of the every file in the project" if you
> prefer that wording.

Not really.  Somebody who is skimming, who reads just the headline
without reading enumeration, would not be able to tell differenes
between a tree and a commit.  Your enumeration lists _what_ is
recorded, the headline I gave you above explains _what_ they are
recorded _for_.

> I think the current phrasing is okay. I also didn't respond to some of the
> phrasing suggestions above if I didn't understand the goal of them.
> Hope that's okay.

If you do not understand, please ask ;-)